A reinforcement learning theory for homeostatic regulation

Reinforcement learning models address animal’s behavioral adaptation to its changing “external” environment, and are based on the assumption that Pavlovian, habitual and goal-directed responses seek to maximize reward acquisition. Negative-feedback models of homeostatic regulation, on the other hand, are concerned with behavioral adaptation in response to the “internal” state of the animal, and assume that animals’ behavioral objective is to minimize deviations of some key physiological variables from their hypothetical setpoints. Building upon the drive-reduction theory of reward, we propose a new analytical framework that integrates learning and regulatory systems, such that the two seemingly unrelated objectives of reward maximization and physiological-stability prove to be identi- cal. The proposed theory shows behavioral adaptation to both internal and external states in a disciplined way. We further show that the proposed framework allows for a unified explanation of some behavioral pattern like motivational sensitivity of different associative learning mechanism, anticipatory responses, interaction among competing motivational systems, and risk aversion

Spectral Statistics of "Cellular" Billiards

For a bounded planar domain $\Omega^0$ whose boundary contains a number of flat pieces $\Gamma_i$ we consider a family of non-symmetric billiards $\Omega$ constructed by patching several copies of $\Omega^0$ along $\Gamma_i$'s. It is demonstrated that the length spectrum of the periodic orbits in $\Omega$ is degenerate with the multiplicities determined by a matrix group $G$. We study the energy spectrum of the corresponding quantum billiard problem in $\Omega$ and show that it can be split in a number of uncorrelated subspectra corresponding to a set of irreducible representations $\alpha$ of $G$. Assuming that the classical dynamics in $\Omega^0$ are chaotic, we derive a semiclassical trace formula for each spectral component and show that their energy level statistics are the same as in standard Random Matrix ensembles. Depending on whether ${\alpha}$ is real, pseudo-real or complex, the spectrum has either Gaussian Orthogonal, Gaussian Symplectic or Gaussian Unitary types of statistics, respectively.Comment: 18 pages, 4 figure

Insecurity for compact surfaces of positive genus

A pair of points in a riemannian manifold $M$ is secure if the geodesics between the points can be blocked by a finite number of point obstacles; otherwise the pair of points is insecure. A manifold is secure if all pairs of points in $M$ are secure. A manifold is insecure if there exists an insecure point pair, and totally insecure if all point pairs are insecure. Compact, flat manifolds are secure. A standing conjecture says that these are the only secure, compact riemannian manifolds. We prove this for surfaces of genus greater than zero. We also prove that a closed surface of genus greater than one with any riemannian metric and a closed surface of genus one with generic metric are totally insecure.Comment: 37 pages, 11 figure

The Simplest Piston Problem II: Inelastic Collisions

We study the dynamics of three particles in a finite interval, in which two light particles are separated by a heavy ``piston'', with elastic collisions between particles but inelastic collisions between the light particles and the interval ends. A symmetry breaking occurs in which the piston migrates near one end of the interval and performs small-amplitude periodic oscillations on a logarithmic time scale. The properties of this dissipative limit cycle can be understood simply in terms of an effective restitution coefficient picture. Many dynamical features of the three-particle system closely resemble those of the many-body inelastic piston problem.Comment: 8 pages, 7 figures, 2-column revtex4 forma

Splay states in finite pulse-coupled networks of excitable neurons

The emergence and stability of splay states is studied in fully coupled finite networks of N excitable quadratic integrate-and-fire neurons, connected via synapses modeled as pulses of finite amplitude and duration. For such synapses, by introducing two distinct types of synaptic events (pulse emission and termination), we were able to write down an exact event-driven map for the system and to evaluate the splay state solutions. For M overlapping post synaptic potentials the linear stability analysis of the splay state should take in account, besides the actual values of the membrane potentials, also the firing times associated to the M previous pulse emissions. As a matter of fact, it was possible, by introducing M complementary variables, to rephrase the evolution of the network as an event-driven map and to derive an analytic expression for the Floquet spectrum. We find that, independently of M, the splay state is marginally stable with N-2 neutral directions. Furthermore, we have identified a family of periodic solutions surrounding the splay state and sharing the same neutral stability directions. In the limit of $\delta$-pulses, it is still possible to derive an event-driven formulation for the dynamics, however the number of neutrally stable directions, associated to the splay state, becomes N. Finally, we prove a link between the results for our system and a previous theory [Watanabe and Strogatz, Physica D, 74 (1994), pp. 197- 253] developed for networks of phase oscillators with sinusoidal coupling.Comment: 27 pages, 12 Figures, submitted to SIAM Journal on Applied Dynamical Systems (SIADS

The boundary integral method for magnetic billiards

We introduce a boundary integral method for two-dimensional quantum billiards subjected to a constant magnetic field. It allows to calculate spectra and wave functions, in particular at strong fields and semiclassical values of the magnetic length. The method is presented for interior and exterior problems with general boundary conditions. We explain why the magnetic analogues of the field-free single and double layer equations exhibit an infinity of spurious solutions and how these can be eliminated at the expense of dealing with (hyper-)singular operators. The high efficiency of the method is demonstrated by numerical calculations in the extreme semiclassical regime.Comment: 28 pages, 12 figure